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(54) Video coder, decoder and transmission system 



(57) A picture coder decodes P frames without hav- 
ing to wait tor the next I frame even il a data loss occurs, 
and ensures quick recovery from degraded picture qual- 
ity. The picture coder, which controls updating of a refer- 
ence picture based on a decoding status signal and a 
frame number sent from a picture decoder, uses an 
acknowledgment signal ( ACK signal) as a decoding sta- 
tus signal. The picture coder has a decoding status 
determination module 109 which determines that a 
decoding error has occurred if the acknowledgment sig- 

Fig . / 



nai is not received within a specified time. The picture 
coder also has a reference frame updating module 110 
for use when it is determined that a decoding error has 
occurred. The reference frame updating module 110 
selects from pictures, which were received correctly 
before the decoding error occurred, one picture for use 
as the reference picture. This makes it possible for the 
picture coder to decode P frames without having to wait 
for the next I frame. 
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Description 

BACKGROUND OF THE INVENTION 

1. Field of the invention 

The present invention relates to a picture coder, 
and more particularly to an improvement in picture qual- 
ity degraded by a data error or a data loss caused dur- 
ing picture data transmission. 

2. Description of the prior art 

Recently, standardization of picture coding meth- 
ods has been under way, with the implementation of 
moving-picture transmission systems in mind. Those 
systems include a videotelephone, videoconferencing, 
and video-on-demand (VOD). ITU-T Recommendation 
H.263 and MPEG (Moving Picture Experts Group) are 
well known as international standardization standards. 
For example, the coding method adopted by the ITU-T 
Recommendation H.263, shown in FIG. 3 t performs 
intra-frame coding on time-sequenced frames (I frames 
a and i) at regular intervals and, at other times, performs 
inter-frame coding for each of P frames (inter-frame 
coded frames, b - h, j - ) with reference to the immedi- 
ately preceding frame to remove temporal redundancy. 
In the following discussion, a frame coded in the intra- 
frame coding method is referred to as an I frame, while 
a frame coded in the inter-frame coding method is 
referred to as a P frame. 

This technology is described in "Standardization of 
Multimedia Coding" by Hiroshi Yasuda, pp. 84 - 97, 
Maruzen, (1991). 

The method proposed by Recommendation H.263, 
which performs the inter-frame coding of each frame by 
referencing the immediately preceding frame, requires 
that all the frames be transmitted in the correct 
sequence. For a telephone line or an ISDN line over 
which data is transmitted after a connection with a part- 
ner is established, data reaches the partner without loss 
and in the correct sequence. 

However, for an Ethernet LAN or an ATM network in 
which data is divided into small units (called packets or 
cells) before it is transmitted, there is a possibility that 
packets are lost or transmitted in an incorrect sequence. 

In general, networks employ a protocol (for exam- 
ple, TCP : Transmission Control Protocol) in which the 
transmitting device sends packets with attached serial 
numbers, and the receiving device rearranges the pack- 
ets in the correct sequence, confirms their arrival, and 
sends requests for the retransmission of non-arriving 
packets back to the transmitting device in order to deal 
with these problems and increase network reliability. 

However, when network operation is unstable and 
packets are dropped frequently, retransmission under 
this type of protocol can cause large cumulative delays 
to build up, which is inappropriate for the real-time trans- 



mission of moving pictures. In some cases, it is prefera- 
ble to display new data, even if that means skipping a 
frame, rather than retransmitting old data, especially 
when new data can be displayed immediately. 
5 Broadcasting and multicasting are schemes which 

send data to a plurality of sites at a time. However, when 
packet dropout occurs during transmission of a packet 
to one of the sites, the above protocol requires that the 
same packet be sent even to those sites which have 
10 received the packet successfully, significantly increas- 
ing the network load. 

Broadcasting and multicasting are therefore per- 
formed using a protocol that does not re-transmit a 
packet, such as the User Datagram Protocol (UDP); as 
is a result, the probability of packet dropout increases. 

In wireless networks, the data error rate or data 
drop-out rate is high not only when data is sent in pack- 
ets but when a line connection is established before 
data is sent. In addition, when the errors exceed the 
20 error-correcting capability of the receiving device, a 
sequence of data items are sometimes discarded to 
receive some other part of data successfully. Data drop- 
outs in wireless networks therefore tend to be larger 
than in wireline networks. 
25 Another problem is that the processing speed of the 
sending device is not always equal to that of the receiv- 
ing device. For example, decoding all the frames on a 
slower receiving device would put much frame data in 
the wait state, causing long delays. This requires the 
30 receiving device to intentionally skip frames. However, 
when there is no decoding data for a frame preceding 
the current frame, as in the inter-frame coding method 
according to the prior art, the current frame cannot be 
decoded and therefore frames cannot be skipped as 
35 intended. 

FIG. 4 shows an example of frame dropout that may 
occur during transmission of frames. When frame e is 
dropped out or cannot be decoded because of slow 
processing, the P frames (f, g, h) cannot be decoded 
40 until the next I frame, i, is received. 

Thus, in order to send all the frames successfully in 
a network in which frame dropout or frame skipping 
occurs frequently, intra-frame coding for all the frames is 
more efficient. However, the problems with this method, 
45 which does not use inter-frame coding, are that there is 
temporal redundancy and that data transmission effi- 
ciency is degraded. 

In addition, in a system where a coder performs 
processing in response to status information sent from a 
so receiving device and, upon detecting a transmission 
error or a decoding error, a negative acknowledgment 
signal (NACK : Negative Acknowledgment Signal) is 
sent, there are following problems. For example, when 
data identification information contained in coded data, 
55 such as a data number, is incorrect, information on an 
incorrect data number may not be sent back to the 
coder. In this case, the coder either ignores the negative 
acknowledgment signal or performs processing on 
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unintended data with an incorrect data number This 
may lead to an unexpected operation. 

To solve the problem of negative acknowledgment 
signals, it is possible to estimate the correct numbers 
from correct data received after the error and to send s 
the acknowledgment signals at a time. However, this 
method prevents acknowledgment signals from being 
sent until the end of incorrect data numbers is detected, 
causing a delay and preventing quick recovery from 
degraded picture quality. 10 

Thus, to successfully transmit moving-picture data 
over a network in which a data dropout occurs fre- 
quently, there has been a long felt need for a picture 
coding system which ensures high coding efficiency 
and quick recovery from degraded picture quality. is 

SUMMARY OF THE INVENTION 

To solve the above problems, a picture coder 
according to the present invention includes coding 20 
means for coding input pictures through predictive cod- 
ing, one at a time, based on a reference picture and for 
outputting coded data, transmission means for transmit- 
ting the coded data coded by the coding means and a 
frame number thereof to a picture decoder, and refer- 25 
ence picture updating means for controlling updating of 
the reference picture based on a decoding status signal 
sent from the picture decoder and an associated frame 
number. The picture coder comprises: 

That is, the picture coder comprises means which 30 
uses an acknowledgment signal (ACK : Acknowledg- 
ment signal) as the decoding status signal and which 
returns the acknowledgment signal when there is no 
transmission or decoding error. The means determines 
that a decoding error has occurred if the acknowledge 35 
signal is not received within a specified time. And, as a 
next reference picture, the reference picture updating 
means selects one of pictures transmitted correctly 
before the decoding error occurred. 

The picture coder according to the present inven- 40 
tion uses a decoder which comprises reception means 
for receiving coded data coded thorough predictive cod- 
ing and a frame number (or frame number and block 
number) of a reference picture of the coded data over a 
transmission line, decoding means for decoding coded 45 
data, received by the reception means, with the use of a 
frame number (or frame number and block number) of 
the reference picture, and decoding status signal trans- 
mission means for outputting to the picture coder a 
decoding status signal representing a status of decoded so 
data decoded by the decoding means and a frame 
number (or frame number and block number) corre- 
sponding to the data. 

Thus, the picture decoder sends decoding status 
information on received coded data to the picture coder ss 
and. during coding, uses one of correctly-decoded 
frames (or blocks) as the reference picture. Therefore, 
the picture decoder can decode a P frame without hav- 



ing to wait for the next I frame. 

In addition, the acknowledgment signal is used as 
the decoding status signal. This enables a correct frame 
number (or frame number and block number) to be 
returned to the coder, allowing the coder to perform 
processing correctly. At the same time, elimination of 
the need to wait for data after an error minimizes a 
delay. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood more fully 
from the detailed description and the accompanying 
drawings below of the preferred embodiments of the 
invention. 

In the drawings: 

FIG. 1 is a block diagram showing an example of a 
functional conf iguration of a picture coder (sending 
device) used in a first embodiment. 
FIG. 2 is a block diagram showing an example of 
detailed functional configuration of a reference 
frame updating module used in the first embodi- 
ment. 

FIG. 3 is a conceptual diagram showing conven- 
tional coding. 

FIG. 4 is a conceptual diagram showing an example 
of coding when a transmission error occurs in a 
conventional system. 

FIG. 5 is a conceptual diagram showing an example 
of coding in the first embodiment. 
FIG. 6 is a conceptual diagram showing an example 
of coding when an ACK signal transmission error 
occurs in the f irst embodiment. 
FIG. 7 is a block diagram showing an example of 
functional configuration of a picture coder (sending . 
device) used in a second embodiment. 
FIG. 8 is a block diagram showing an example of 
detailed functional configuration of a reference 
frame updating module used in the second embod- 
iment. 

FIG. 9 is a conceptual diagram showing an example 
of coding when a transmission error occurs in the 
second embodiment. 

FIG. 10 is a conceptual diagram showing an exam- 
ple of coding when a transmission error occurs in 
an active mode. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

(A) First embodiment 

Referring to the attached drawings, there is shown 
a configuration, operation, and effect of a moving-pic- 
ture coder (sending device) used in the first embodi- 
ment. 

The moving -picture coder according to the present 
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invention updates a reference picture on a frame basis 
or on a block basis, where a block is one partition of a 
frame. When picture data is processed on a frame 
basis, a decoding status signal is returned and the ref- 
erence picture is updated each time a frame is updated; 
when picture data is processed on a block basis, the 
decoding status signal is returned and the reference 
picture is updated each time a block is updated. The 
processing and configuration described below apply to 
frames and blocks, although only frames are mentioned 
in some part of the description 

The picture coder (sending device) in the first 
embodiment is characterized in that new means is pro- 
vided to check the status of the acknowledgment (ACK) 
signal. That is, the means determines that no error has 
occurred on a frame or a block for which the acknowl- 
edgment signal is received and that a decoding error 
has occurred on a frame or a block for which the 
acknowledgment signal is not received. The picture 
coder updates reference frames according to the ACK 
signal only. 

A functional configuration of the moving-picture 
coder (sending device) used in the first embodiment is 
shown in FIG. 1 . 

As shown in FIG. 1, a moving -picture coder 100 
comprises a moving-picture input module 101, a coding 
module 102, a decoding module 103 , a frame memory 
module 104, a reference frame memory module 105. a 
intra/irrter determination module 106, a coded data 
transmitting module 107, an acknowledgment signal 
receiving module 108, a decoding status determination 
module 109, and a reference frame updating module 
110. 

The moving-picture input module 101 passes mov- 
ing-picture data from an input source, such as a video 
camera, to the coding module 102, one frame at a time. 

The coding module 102 codes entered frame data, 
one frame at a time or one block at a time, and passes 
coded data to the decoding module 103 and the coded 
data transmitting module 107. When coding data, the 
coding module 102 switches processing between intra- 
frame coding and inter-frame coding according to an 
instruction from the intra/inter determination module 
106 , where "intra" refers to intra-frame coding and 
"inter" refers to inter-frame coding. When the coding 
module 102 performs inter-frame coding, it references 
frame data stored in the reference frame memory mod- 
ule 105. 

The decoding module 103 decodes coded data and 
writes decoded data, as well as frame numbers, into the 
frame memory module 104. 

The frame memory module 104 is a memory 
medium, such as a memory circuit, which memorizes 
data for each frame. Each time a reference frame is 
updated by the reference frame updating module 110, 
unnecessary data is cleared from the frame memory 
module 104.. 

The reference frame memory module 105, a mem- 



ory medium such as a memory circuit which contains 
reference frame data for use in inter-frame coding, is 
updated by the reference frame updating module 110. 
The intra/inter determination module 106 selects 

5 which coding to use: intra-frame coding or inter-frame 
coding. Normally, intra-frame coding is selected at reg- 
ular intervals (e.g., once every thirty frames) and, at 
other times, inter-frame coding is selected; when the 
intra/inter determination module 106 receives a refresh 

10 signal from a receiving device (moving-picture decoder 
not shown in the drawing), it forces intra-frame coding 
and passes information to that effect to the coding mod- 
ule 102 and the reference frame updating module 110 
The coded data transmitting module 107 sends 

is coded moving-picture data to one or more receiving 
devices (moving-picture decoders). The coded data 
transmitting module 107 multiplexes control information, 
such as an intra/inter flag or a reference frame number 
onto the data; it also multiplexes sending device infor- 

20 mation and receiving device information, as necessary, 
onto the data. 

The acknowledgment signal receiving module 108 
receives the acknowledgment signal (decoding status 
signal), which indicates that which frames (or blocks) 

25 have been decoded, from the receiving device (moving- 
picture decoder), and passes the received decoding 
status signal to the reference frame updating module 
110. 

The decoding status determination module 109 

30 determines that a frame (or a block) for which the 
acknowledgment signal (ACK signal) is received within 
a specified time has no decoding error and that a frame 
(or a block) for which the acknowledgment signal is not 
received within a specified time has a decoding error. It 

35 then passes the acknowledgment status and the frame 
number (or frame number and block number) to the ref- 
erence frame updating module 110. 

The reference frame updating module 110 updates 
the reference frame, responsive to the instructions 

40 received from the intra/inter determination module 106 
and the decoding status determination module 109. The 
reference frame updating module 110 usually updates 
the reference frame each time it receives a frame; when 
a decoding error occurred, the module returns the refer- 

45 ence frame to a most recent frame (or block) for which 
acknowledgment signal was received. Upon receiving 
an intra-frame coding instruction from the intra/inter 
determination module 106, the reference frame updat- 
ing module 110 copies the data of the current frame 

so from the frame memory module 104 to the reference 
frame memory module 105 and deletes all the frame 
data from the frame memory module 104. 

The reference frame updating module 1 1 0 is further 
divided into detailed functional blocks. FIG. 2 is a 

55 detailed functional configuration diagram of the refer- 
ence frame updating module 110. As shown in FIG. 2. 
the reference frame updating module 110 comprises a 
frame number setting module 111, a last-received- 
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frame number detecting module 112, a candidate 
number deleting module 113, a reference frame candi- 
date number recording module 114, and a data copy 
module 115. 

When the frame number setting module 111 
receives information indicating that a decoding error 
has occurred from the decoding status determination 
module 109, it passes the incorrect-frame number (or 
frame number and block number) to the last-received- 
frame number detecting module 1 12 and the candidate 
number deleting module 113. Upon receiving a last- 
received-frame number from the last-received-frame 
number detecting module 112, the frame number set- 
ting module 1 1 1 sets up the last-received-frame number 
as the number of frame (or the number of frame and the 
number of block) to'be copied, and passes the number 
to the data copy module 1 15. 

When the frame number setting module 111 
receives information indicating that "there is no candi- 
date number/ it determines that no error occurred and 
suspends processing. 

When the frame number setting module 111 
receives information indicating that no decoding error 
has occurred from the decoding status determination 
module 109, it passes the correct-frame number (or 
frame number and block number) to the candidate 
number deleting module 113. 

The last-received-frame number detecting module 
112 searches a candidate number list stored in the ref- 
erence frame candidate number recording module 114 
to check to see if an incorrect-frame number received 
from the frame number setting module 1 11 is recorded 
in the list. 

When the incorrect-frame number received from 
the frame number setting module 1 1 1 is recorded in the 
candidate frame number list the last-received-frame 
number detecting module 112 finds a last-received- 
frame number and returns it to the frame number setting 
module 111. On the other hand, when the incorrect- 
frame number received from the frame number setting 
module 111 is not recorded in the candidate frame 
number list, the last-received-frame number detecting 
module 112 returns information indicating that "there is 
no candidate number" to the frame number setting mod- 
ule 111. 

When the candidate number deleting module 113 
receives an incorrect-frame number (or frame number 
and block number) from the frame number setting mod- 
ule 1 1 1 . it deletes the incorrect-frame number (or frame 
number and block number) and the frame numbers 
added later than the frame number from the candidate 
frame number list stored in the reference frame candi- 
date number recording module 114. This keeps only 
correct-frame numbers in the reference frame candidate 
number recording module 114. At this time, it is also 
possible to delete frame data corresponding to the 
frame number from the frame memory module 104. 

When the candidate number deleting module 113 



receives a correct-frame number (or frame number and 
block number) from the frame number setting module 
1 1 1 , it records the number in the reference frame candi- 
date number recording module 114 as a last-received- 
5 frame number, tt also deletes the number (or frame 
number and block number) and the frame numbers ear- 
lier than the frame number from the candidate frame 
number list recorded in the reference frame candidate 
number recording module 114. At this time, it is also 
70 possible to delete frame data corresponding to the 
deleted frame number from the frame memory module 
104. 

The reference frame candidate number recording 
module 1 14 manages the numbers of frame data stored 
75 in the frame memory module 104. It records a last- 
received-frame number, which is the number of a frame 
to be used as a reference frame, and a list of candidate 
frame numbers. A list may be created for each block. 
Thus, each time latest frame data coded by the coding 
20 module 102 is recorded in the frame memory module 
104, the reference frame candidate number recording 
module 114 adds the frame number to the candidate 
list. The candidate frame number list, which is in the ref- 
erence frame candidate number recording module 114 
25 in the above description, may be recorded in the frame 
memory module 104 as header information. 

The data copy module 115 usually copies latest 
frame data from the frame memory module 104 to the 
reference frame memory module 105. Upon receiving a 
30 last-received-frame number from the frame number set- 
ting module 1 1 1 , the data copy module 1 15 copies the 
corresponding frame data from the frame memory mod- 
ule 104 to the reference frame memory module 105. 
Next, the operation of the picture coder (sending 
35 device) used in the first embodiment is described. 

The moving-picture coder checks the decoding sta- 
tus signal, which is returned from a moving-picture 
decoder, to switch a reference picture used in inter- 
frame predictive coding. The moving-picture coder uses 
40 the acknowledgment signal (ACK signal) as the decod- 
ing status signal. When a decoding error occurs, this 
signal is not returned. 

The moving-picture coder updates the reference 
picture each time it receives a frame while no decoding 
45 error occurs. This is the same coding method as that 
proposed by H.263 in which the immediately-preceding 
frame is always referenced during coding. When a 
decoding error is detected, the last frame received cor- 
rectly is used as the reference picture. 
so FIG. 5 shows an example of operation that is per- 
formed by the moving-picture coder which has the refer- 
ence frame updating module built-in. For simplicity, 
each frame is composed of one block. For a multiple- 
block frame, the coder does the same operation for 
55 each block. 

While no decoding error occurs, the moving-picture 
coder updates the reference frame each time it receives 
a frame. When the ACK signal arrives, the coder 
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records the number of the frame (or block) as the last- 
received-frame number. For example, when frame e is 
coded, the last- received-frame number is frame a 
When a transmission error occurs on frame d, the mov- 
ing-picture decoder cannot decode frame d and there- 
fore does not return the ACK signal for frame d. When 
the ACK signal is not received, the moving-picture coder 
detects that a decoding error has occurred on frame d 
and changes the reference frame to the frame corre- 
sponding to last-received-frame number (frame c) and 
codes frame f. 

By doing so, the moving -picture decoder can 
decode the next frame f and the following frames, 
although frame e cannot be decoded because it is 
already coded and transmitted before the decoding 
error is detected. This allows the system to quickly 
recover from a decoding error caused by a transmission 
error. 

When a decoding error is detected on frame d, the 
moving-picture coder deletes the numbers of frame d 
and all the later frames (only frame e in the example in 
the figure) from the reference frame candidate number 
list. This prevents frame d which is incorrect and frame 
e which references frame d and therefore causes a 
decoding error from being selected as a reference pic- 
ture. In addition, as explained below, decoding error 
processing for frame e may be omitted. At this time, it is 
possible to delete frame data corresponding to the 
deleted numbers from the frame memory module 104. 
This enables a small-memory system to delete data 
which becomes unnecessary, eliminating the possibility 
of clearing necessary data due to a shortage of memory 
and thereby ensuring coding effidency. 

The ACK signal is not returned for frame e because 
it cannot be decoded by the moving-picture decoder. 
Therefore, the moving-picture coder detects a decoding 
error on frame e. If, at this time, the reference frame is 
changed to a previous frame (c) for frame e, frame g 
also references frame c, with degraded coding effi- 
. ciency. However, because frame e is already deleted 
from the reference frame candidate list, information indi- 
cating "there is no candidate number" is returned. 
Therefore, the reference frame is not returned to a 
frame with the tast-received-frame number, with no deg- 
radation in coding efficiency. 

FIG. 6 shows an example of coding when an ACK 
signal error occurs. When an ACK transmission error 
occurs on frame d, the moving-picture coder detects 
that a decoding error has occurred on frame d, changes 
the reference frame to a frame (c) with the last-received- 
frame number, and then codes frame f. Although the 
moving-picture decoder has received frame d correctly 
the reference frame is changed to a previous frame and 
therefore the coding efficiency of frame f is degraded to 
some degree. However, it should be noted that all the 
frame are decoded and little or no effect is found on 
other frames. In a system where the negative acknowl- 
edgment signal (NACK signal) is used as the decoding 



status signal, a NACK signal error generates a frame 
that cannot be decoded, causing somewhat serious 
effects. On the other hand, in a system where the 
absence of the ACK signal indicates a decoding error, 

5 an ACK signal transmission error would minimize cod- 
ing efficiency degradation to such a level that almost 
every person cannot notice. 

As described above, when no transmission error 
occurs in a system of the first embodiment, the refer- 

io ence frame is updated each time a frame is received. 
Thus, the coding efficiency is the same as that of other 
coding methods such as that proposed by ITU-T Rec- 
ommendation H.263. 

When a transmission error occurs, decoding correct- 
is ness of a frame already coded cannot be guaranteed 

but the subsequent frames can be decoded before the 

next I frame arrives and therefore coding efficiency is 

not degraded. 

In addition, use of the acknowledgment signal (ACK 
20 signal) as the decoding status error makes it possible to 

transmit frame numbers and block numbers correctly 

and to eliminate the need to wait for correct data after 

an error, thus minimizing the delay. 

Even rf an ACK signal transmission error occurs, coding 
25 efficiency is somewhat degraded but all the frames are 

decoded. 

Another advantage is that, when continuous decod- 
ing errors are detected, the reference frame candidate 
list used in this embodiment bypasses the processing of 
30 changing the reference frame to a previous frame when 
the second error occurs, ensuing coding efficiency. 

Variations of the first embodiment are: 

(1) In the above embodiment, the reference frame 
35 updating module 110 copies frame data from the 

frame memory module 104 to the reference frame 
memory module 105 when intra-frame coding is 
performed. This need not always be done. For 
example, the intra/inter determination module 106 

40 may send an intra-frame coding request signal to 
the decoding module 103 and. at the same time, 
the decoding module 103 may write frame data 
directly into the reference frame memory module 
105 during intra-frame coding. This enables copy 

45 processing to be bypassed and makes the configu- 
ration simple. 

(2) In FIG. 1 , the reference frame memory module 
105 and the frame memory module 104 are sepa- 
rate modules. This configuration is not always 

so required. Frame data may be stored in a memory 
module, with the reference frame being pointed to 
by a pointer. This configuration eliminates the need 
for copying data from the frame memory module 
104 to the reference frame memory module 105 

55 during intra-frame coding; simply moving the 
pointer points to the reference frame, thereby mak- 
ing the circuit configuration simple. 
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(B) Second embodiment 

Referring to the attached drawings, there is shown 
a configuration and operation of a moving-picture coder 
(sending device) used in the second embodiment. 

The following describes a reference frame update 
method which uses an acknowledgment signal (ACK 
signal) but which is different from a method used in the 
first embodiment. A moving-picture coder used in the 
second embodiment uses only the ACK signal and 
switches the reference frame update method between 
the method different from the first embodiment and the 
method used in the first embodiment according to the 
status of a transmission line. 

FIG. 10 shows an example of operation of the 
update method different from that used in the first 
embodiment. This method is characterized in that a ref- 
erence picture is updated only when the ACK signal is 
received but not when the ACK signal is not received. 

For example, for each of frames (a, b, c), the ACK 
signal is sent from the receiving device to indicate that it 
has been received correctly. Each time the ACK signal 
is received, the reference frame is switched and the next 
frame is coded. 

When a error occurs during transmission of frame 
d, the ACK signal is not sent and therefore frame d is not 
used as a reference frame. Instead, frame c. which was 
already received correctly, is used as a reference frame 
and the next frame f is coded. 

This update method always performs coding with a 
correctly-received reference frame as the reference 
frame. Therefore, the receiving device can decode a P 
frame without having to wait for an I frame that will be 
sent later, with little or no degradation in coding effi- 
ciency even when a frame is lost However, if the round 
trip delay in the period of time from when the sending 
device sends coded data to the time the ACK signal is 
received is large, the ACK signal for the previous frame 
is not received before the coding of the next frame is 
started. This means that t even when there is no trans- 
mission error, a frame that is two or more frames before 
the current frame is referenced and so the coding effi- 
ciency is more or less degraded. 

The update method shown in FIG. 10 updates the 
reference frame when no transmission error occurs, 
while the update method shown in the first embodiment 
updates the reference frame when a transmission error 
occurs. In the following description, the former (the 
update method shown in FIG. 10) is referred to as an 
Active mode, while the latter (the update method shown 
in the first embodiment) as a Negative mode. The Active 
mode is effective when the transmission status is good; 
the Negative mode is effective when the transmission 
status is bad. Thus, switching these two modes accord- 
ing to the transmission status ensures high coding effi- 
ciency. 

The moving-picture coder (sending device) used in 
the second embodiment comprises a transmission sta- 



tus determination module 203 which determines the 
status of the transmission line with the use of the decod- 
ing status signal, an update method switching module 
204 which switches the update mode between the 
5 Active mode and the Negative mode according to the 
transmission line status, and a reference frame adaptive 
updating module 202 which updates the reference 
frame adaptive!/ 

When the transmission line status is good, the Neg- 
10 ative mode is enabled to update the reference frame 
each time a frame is received; in this case, only when a 
transmission error occurs, the reference frame is 
returned to the last reference frame that was positively 
acknowledged. On the other hand, when the transmis- 
75 sion line status is bad, the Active mode is enabled; in 
this case, when a frame is acknowledged, the reference 
frame is updated. 

FIG. 7 is a functional configuration diagram of a 
moving -picture coder (sending device) in the second 
20 embodiment. Note that, for a module in FIG. 7 with a 
function equivalent to that of a module shown in FIG. 1 , 
the corresponding number is used and no description is 
given here. The following describes only new modules: 
decoding status determination module 201. reference 
frame adaptive updating module 202. transmission sta- 
tus determination module 203. and update method 
switching module 204. 

The decoding status determination module 201, 
like the decoding status determination module 109, 
determines that a frame (or a block) for which the 
acknowledgment signal (ACK signal) is received within 
a specified time has no decoding error and that a frame 
(or a block) for which the acknowledgment signal is not 
received within a specified time has a decoding error, 
based on the decoding status signal from the acknowl- 
edgment signal receiving module 108. It then passes 
the acknowledgment status and the frame number (or 
block number) to the reference frame updating module 
110 and the transmission status determination module 
203. 

The reference frame adaptive updating module 202 
updates the reference frame according to instructions 
from the intra/inter determination module 106, decoding 
status determination module 201, and update method 
45 switching module 204. 

There are two reference frame update methods: 
Active mode and Negative mode. In the Active mode, a 
frame (or block) on which no decoding error occurred is 
used as the reference frame picture; in the Negative 
so mode, the reference frame picture is updated each time 
a frame (or block) is received and. when a decoding 
error occurs, the reference frame is returned to the last 
frame on which no decoding error occurred. 

The reference frame adaptive updating module 202 
55 switches the mode between the Active mode and the 
Negative mode according to an instruction from the 
update method switching module 204. 

For example, upon receiving information indicating 
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"there is no decoding error" from the decoding status 
determination module 201 in the Active mode, the refer- 
ence frame adaptive updating module 202 copies frame 
data (or block data) from the frame memory module 104 
to the reference frame memory module 105. At this 
time, data corresponding to numbers earlier than the 
frame number (or frame number and block number) may 
be deleted from the frame memory module 104. When 
information indicating that "there is a decoding error" is 
received, the no operation is performed. At this time, the 
frame number (or block number) may be deleted from 
the reference frame candidate number list, and data 
corresponding to the number may be deleted from the 
frame memory module 104. 

In the Negative mode, the reference frame adaptive 
updating module 202 usually copies frame data (or 
block data) from the frame memory module 104 to the 
reference frame memory module 105 each time frame 
data (or block data) is received. The module does this 
operation in synchronization with the coding module 
102. When information indicating that "there is no 
decoding error" is received from the decoding status 
determination module 201, the last-received-frame 
number (or frame number and block number) which is 
the number of the last frame positively acknowledged is 
updated. At this time, data earlier than the last-received- 
frame number may be deleted from the frame memory 
module 104. When information indicating "there is a 
decoding error" is received, frame data (or block data) 
corresponding to the last-received-frame number is 
copied from the frame memory module 104 to the refer- 
ence frame memory module 105 in order to change the 
reference frame to the previous frame with that number. 

Upon receiving an intra-frame coding request from 
the intra/inter determination module 106, the reference 
frame adaptive updating module 202 copies current 
frame data from the frame memory module 104 to the 
reference frame memory module 105 and deletes all 
frame data from the frame memory module 104. 

The transmission status determination module 203 
determines the current transmission status based on 
the decoding status information received from the 
decoding status determination module 201, and then 
passes the result to the update method switching mod- 
ule 204. The transmission status is either "good* or 
"bad." For example, if a transmission error was detected 
in M frames (or M blocks) out of the last N frames (or N 
blocks), the status is "bad"; otherwise (N ^ M), the sta- 
tus is "good." 

In addition, if a transmission error occurs in consec- 
utive L frames (or L blocks), the status is "bad." There 
may be a plurality of determination criteria at the same 
time. In that case, the status is "good" if all the determi- 
nation results are good, and is "bad" if one of determi- 
nation result is "bad." The status may be determined to 
be "bad" only if all the results are "bad." 

The update method switching module 204 switches 
the update method to be used by the reference frame 



adaptive updating module 202, based on the transmis- 
sion status determined by the transmission status deter- 
mination module 203. For example, when the 
determination result is "good," the module switches the 

5 mode to the Negative mode; when the determination 
result is "bad," the module switches the mode to the 
Active mode. 

The following describes the function of the refer- 
ence frame adaptive updating module 202 with refer- 

w ence to a detailed functional block diagram. 

Fig. 8 is an example of detailed functional configu- 
ration of the reference frame adaptive updating module 
202. Note that, for a module in FIG. 8 with a function 
equivalent to that of a module shown in the detailed 

75 configuration of the reference frame updating module 
1 10 in the first embodiment, the corresponding number 
is used and no description is given here. The following 
describes only a frame number adaptive setting module 
205 which is a new module. 

20 The operation of the frame number adaptive setting 
module 205 depends on the update mode that is 
switched by the update method switching module 204. 

For example, in the Active mode, when the frame 
number adaptive setting module 205 receives inforrna- 

25 tion indicating that there is no decoding error from the 
decoding status determination module 201, it sets up a 
frame number (or frame number and block number), 
which is sent with that information, as the number of a 
frame to be copied and passes the number to the data 

30 copy module 1 15. At the same time, the frame number 
adaptive setting module 205 passes the frame number 
(or frame number and block number) of the positively 
acknowledged frame to the candidate number deleting 
module 113. 

35 When the frame number adaptive setting module 
205 receives from the decoding status determination 
module 201 a determination result indicating that a 
decoding error has occurred, it passes the number of 
incorrect frame (or frame number and block number) to 

40 the candidate number deleting module 113. 

In the Negative mode, when the frame number 
adaptive setting module 205 receives from the decoding 
status determination module 201 a determination result 
indicating that a decoding error has occurred, it passes 

45 the number of the incorrect frame (or frame number and 
block number) to the last-received-frame number 
detecting module 1 12 and the candidate number delet- 
ing module 113. When the frame number adaptive set- 
ting module 205 receives a last-received-frame number 

so from the last-received-frame number detecting module 
112, it sets up the received last-received-frame number 
as the number of frame to be copied as the reference 
picture and then passes the number to the data copy 
module 115. 

55 When the frame number adaptive setting module 
205 receives information indicating that "there is no can- 
didate number" from the last-received-frame number 
detecting module 112, it determines that no decoding 
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error occurred and suspends processing. 

When the frame number adaptive setting module 
205 receives from the decoding status determination 
module 109 a determination result indicating that there 
is no decoding error, it passes the number of correct s 
frame (or frame number and block number) to the can- 
didate number deleting module 113. 

The moving-picture coder (sending device) used in 
the second embodiment is described below. 

In the second embodiment, the sending device io 
checks the transmission status through the use of an 
acknowledgment signal sent from a receiving device. 
Inter-frame coding is performed using one of reference 
frame update modes that is selected according to the 
transmission status. ' 5 

Operation of a receiving device does not depends 
on the mode. That is, the receiving device simply sends 
an acknowledgment signal (ACK signal) to the sending 
device. As described in the first embodiment, the receiv- 
ing device updates a reference frame based on a refer- 20 
ence frame number and a block number multiplexed on 
coded data, and repeats decoding. 

The sending device checks the status of the trans- 
mission line through the use of the ACK signal and, 
depending upon the status, switches the update mode. 25 

Normally, the sending device updates the reference 
frame picture in the Negative mode and, based on the 
updated reference frame, performs picture coding. 

The transmission status determination module 203 
continually monitors the transmission status using one 30 
or more criteria. Upon detecting that the status is "bad," 
the mode is switched the mode from the Negative mode 
to the Active mode that is provided for use when. the 
transmission status is bad. In the example in FIG. 9, the 
transmission status is determined to be "bad" after two 35 
consecutive transmission errors. 

That is, when a transmission error occurs on frame 
d, frame e cannot be decoded because frame d could 
not be decoded successfully. In this case, the ACK sig- 
nal cannot be received two consecutive times and 40 
therefore the transmission status is determined to be 
"bad.'' Then, the mode is changed to the Active mode 
when the next frame, g, is coded while the reference 
picture, which is not yet updated, is still frame c. 

As described above, when the reference frame 45 
update mode is changed to the Active mode, the refer- 
ence frame is updated when the ACK signal is received. 
When the ACK signal is not received, the reference 
frame is not updated. 

When the transmission status determination mod- sc 
ule 203 determines in the Active mode that the trans- 
mission status is "good", the mode returns to the 
Negative mode, in FIG. 9. the transmission status is 
determined to be "good" when four consecutive correct 
frames are received. 5i 

That is, when the ACK signal is received for four 
consecutive frames (I, m, n, and o). the transmission 
status is determined to be "good." At this point, the 



mode changes to the Negative mode and coding is per- 
formed while referencing the immediately-preceding 
frame p. 

Thus, while there is no decoding error, the refer- 
ence frame picture is updated in the Negative mode 
each time a frame is received. This does not degrade 
coding efficiency even when the transmission delay is 
large. Another problem with the Negative mode is that a 
frame immediately following an incorrect frame cannot 
be decoded. However, as long as a decoding error 
occur less frequently, degradation of picture quality is 
not so large; in fact, because the picture quality during a 
non-error period is dominant in the Negative mode, pic- 
ture quality degradation is not large. 

On the other hand, in the Active mode which is 
used when a decoding error occurs frequently, all 
frames are decoded with no large degradation in picture 
quality. In particular, when a decoding error occurs 
repeatedly, coding in the Active mode gives a result 1 - 
1.5 dB better than the result of the conventional coding 
method or the Negative mode. 

In the second embodiment, the coder codes mov- 
ing-picture data by referencing a successfully-received 
frame without waiting for the next I frame, as in the first 
embodiment, ensuring good coding efficiency. In addi- 
tion, the ability to adaptively switching the reference 
frame update mode according to the transmission sta- 
tus gives optimal coding efficiency. 

While there is no transmission error, the reference 
frame picture is updated each time a frame is received. 
Therefore, coding efficiency is not decreased even 
when the round trip delay is large. When a transmission 
error occurs less frequently, the Negative mode is used 
to prevent picture quality from being degraded. When a 
transmission error occurs frequently, the Active mode, 
which stables picture quality with no sudden degrada- 
tion, is used to maintain picture quality. 

In either update mode, only the ACK signal is used 
for the decoding status signal. This decreases the com- 
plexity involved in mode switching. In addition, the ability 
to send a correct frame number or a correct block 
number eliminates the need to wait for correct data after 
an error, minimizing delays. Because the sending 
device uses the decoding status signal to check the 
transmission status in order to determine whether the 
update module must be changed, no additional informa- 
tion is needed to check the transmission status. 

Operation of the receiving device does not depend 
on the update mode of the reference frame. Mode 
> switching is performed only on the sending device, and 
information on mode switching need not be sent to the 
receiving device. This does not require the re-configura- 
tion of the receiving device but prevents additional 
transmission load. It should be noted that mode switch- 
5 ing is performed as a result of a change in the transmis- 
sion line status and that information exchange between 
the sending device and the receiving device requires 
error correction processing. It includes error correction 



9 



.CO 



17 



EP 0 851 685 A2 



18 



codes, error protection, or retransmission. Thus, elimi- 
nation of the need to send switching information results 
in an increase in system operation efficiency. 
Variations of the second embodiment are: 

(1) Although, in the second embodiment, the trans- 
mission status is one of two stages, either "good" or 
"bad," and therefore there are two update modes. 
There may be three or more transmission status 
modes. For example, a new stage "extremely bad" 
may be added. If there is an error in almost all con- 
secutive pieces of data, coding may be done at a 
lower level to reduce the amount of coding and to 
reduce the probability of errors. 

(2) Although only the sending device performs 
mode switching and no mode-switching information 
is sent to the receiving device in the second embod- 
iment, the present invention is not limited to this 
configuration. Mode-switching information may be 
sent to the receiving device to allow it, for example, 
to clear the frame memory module. For example, in 
the Active mode, data that is earlier than the frame 
(or block) selected for the reference frame may be 
deleted. 

(3) In the second embodiment, mode switching may 
be done by the receiving device. The acknowledg- 
ment signal is used to detect an error ratio only on 
a frame basis or on a block basis, while the receiv- 
ing device may detect an error ratio on the trans- 
mission line, more flexibly, on a bit basis with the 
use of the CRC. The receiving device may specify a 
mode based on a detected error ratio, or may send 
a detected error ratio to the sending device to allow 
it to perform mode switching based on the received 
error ratio. 

(4) In the second embodiment, the transmission 
status is determined to be bad if an error is 
detected in L or more consecutive frames or in L or 
more consecutive blocks. It is also possible to 
determine that the transmission status is bad when 
a decoding error occurs consecutively in the same 
block of a plurality of frames continuously 

(5) In the second embodiment, the transmission 
status is determined by the number of decoding 
errors detected through the decoding status signal, 
and the mode is switched automatically according 
to the determined status. The transmission status 
may also be determined manually. For example, a 
user on the sending device may switch the update 
mode or may send a switching signal to allow a user 
on the receiving device to switch the mode. 

(C) Other embodiments 

(1) The moving-picture coder in each of the above 
embodiments may be implemented as a hardware 
unit or a software program. 

(2) In each of the above embodiments, transmis- 



sion data is moving-picture data. Transmission data 
may be voice data or binary data. 

As described above, according to the present 

5 invention, the picture decoder sends the decoding sta- 
tus signal and a frame number (or frame number and 
block number) to the picture coder. The picture coder 
controls the update of a reference picture based on the 
decoding status signal and the frame number (or frame 

io number and block number). If a transmission line over 
which coded data is transmitted is low in error frequency 
and high in transmission quality, high picture quality is 
ensured. Conversely, if the line is high in error frequency 
and low in transmission quality, P frames may be 

is decoded before the next I frame arrives. 

While there has been described what are at present 
considered to be preferred embodiments of the inven- 
tion, it will be understood that various modifications may 
be made thereto, and it is intended that the appended 

20 claims cover ail such modifications as fall within the true 
spirit and scope of the invention. 

Claims 

25 1. A picture coder which incjudes coding means for 
coding input pictures through predictive coding, one 
at a time, based on a reference picture and for out- 
putting coded data; transmission means for trans- 
mitting the coded data coded by the coding means 

30 and a frame number thereof to a picture decoder; 
and reference picture updating means for control- 
ling updating of the reference picture based on a 
decoding status signal sent from the picture 
decoder and an associated frame number, said pic- 

35 ture coder comprising: 

decoding status determination means which 
uses an acknowledgment signal as the decod- 
ing status signal and which determines that a 
40 decoding error has occurred rf the acknowl- 

edge signal is not received within a specified 
time, 

wherein, as a next reference picture, the refer- 
ence picture updating means selects one of 
45 pictures transmitted correctly before the decod- 

ing error occurred. 

2. A picture coder as set forth in claim 1, wherein the 
reference picture updating means divides each 
so frame of the input picture into a plurality of blocks 
and updates the reference picture using the frame 
number of the frame and a block number of each of 
subdivided blocks. 

55 3. A picture coder as set forth in claim 1, wherein the 
reference picture updating means records a 
number of the frame for which the acknowledgment 
signal is received as a last-received number and, 
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upon detecting the decoding error, sets up the 
frame or the block with the last-received number as 
the reference picture. 

A picture coder as set forth in claim 2, wherein the s 
reference picture updating means records a 
number of the frame for which the acknowledgment 
signal is received as a last-received number and, 
upon detecting the decoding error, sets up the 
frame or the block with the last-received number as 10 
the reference picture. 

. A picture coder as set forth in claim 1, wherein the 
reference picture updating means comprises: 

75 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame 20 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 

wherein, the picture with the frame number 
stored in the frame number storage module is 25 
selected as the reference frame. 

). A picture coder as set forth in claim 2, wherein the 
reference picture updating means comprises: 

30 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame 35 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is 40 
selected as the reference frame. 

7. A picture coder as set forth in claim 3, wherein the 
reference picture updating means comprises: 

45 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame so 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is 55 
selected as the reference frame. 

8. A picture coder as set forth in claim 4, wherein the 



reference picture updating means comprises: 

a frame number storage module for storing the 
frame number of the input picture, coded by the 
coding means, as a candidate of the reference 
picture; and 

a deleting module for deleting the frame 
number on which the decoding error occurred 
and frame numbers of input pictures coded 
after the decoding error, 
wherein, the picture with the frame number 
stored in the frame number storage module is 
selected as the reference frame. 

9. A picture coder as set forth in claim 1 , further com- 
prising: 

status determination means for determining a 
transmission line status based on the decoding 
status signal and the frame number; and 
update mode switching means for switching 
and controlling, in response to a determination 
result of the transmission line status determi- 
nation means, an update method of the refer- 
ence image updated by the reference picture 
updating means. 

10. A picture coder as set forth in claim 2. further com- 
prising: 

status determination means for determining a 
transmission line status based on the decoding 
status signal and the frame number; and 
update mode switching means for switching 
and controlling, in response to a determination 
result of the transmission line status determi- 
nation means, an update method of the refer- 
ence image updated by the reference picture 
updating means. 

11. A picture coder as set forth in claim 9, wherein, 
when the transmission line status determination 
means determines that the transmission line status 
is good, the update method switching means 
switches the update method from the update 
method by which the reference picture is updated 
when there is no decoding error to the update 
method by which the reference picture is changed 
to one of previous pictures when there is the decod- 
ing error. 

12. A picture coder as set forth in claim 10, wherein, 
when the transmission line status determination 
means determines that the transmission line status 
is good, the update method switching means 
switches the update method from the update 
method by which the reference picture is updated 
when there is no decoding error to the update 



11 



21 



EP 0 851 685 A2 



method by which the reference picture is changed 
to one of previous pictures when there is the decod- 
ing error. 

13. A picture coder as set forth in claim 9, wherein, 5 
when the transmission line status determination 
means determines that the transmission line status 

is bad. the update method switching means 
switches the update method from the update 
method by which the reference picture is changed w 
to one of previous pictures when there is the decod- 
ing error to the update method by which the refer- 
ence picture is updated when there is no decoding 
error. 

15 

14. A picture coder as set forth in claim 10, wherein, 
when the transmission line status determination 
means determines that the transmission line status 
is bad, the update method switching means 
switches the update method from the update 20 
method by which the reference picture is changed 

to one of previous pictures when there is the decod- 
ing error to the update method by which the refer- 
ence picture is updated when there is no decoding 
error. 25 

15. A method of transmitting coded picture data com- 
prising 

coding input pictures sequentially by predictive 
coding, based on a reference picture, so as to 
produce coded data for the pictures succes- 
sively; 

transmitting the coded data to a picture 
decoder; and 

controlling updating of the reference picture 
based on a decoding status signal sent from 
the picture decoder and an associated frame 
number, characterised by 
using as the decoding status signal, acknowl- 
edgment signals (ACK) from the picture 
decoder which respectively indicate successful 
decoding of the data for an individual one of the 
pictures by the decoder, thereby indicating that 
a decoding error has occurred if not received 
within a specified time, and 
selecting as the reference picture, one of pic- 
tures transmitted correctly before the decoding 
error occurred. 

16. A picture coder operative in accordance with the 
method claimed in claim 15. 

1 7. A picture decoder operative to decode coded pic- 
ture data produced in accordance with the method 55 
claimed in claim 15, and to produce said acknowl- 
edgement signals in response to successful decod- 
ing of the coded data for the pictures respectively. 
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